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ABSTRACT 

As part of a symposium on "A Plan for the 
Comprehensive Evaluation of College Teaching," this paper focuses on 
the third of three suggested levels of an evaluation matrix, specific 
feedback data aimed at pinpointing reasons for and solving problems 
with teaching and classroom activities. Methods of getting specific 
feedback data from students and colleagues and of providing 
assistance to improve instruction are examined. Such data are usually 
collected to calrify and explicate problems identified by Level 2 
evaluation. Student input to Level 3 evaluation may take several 
forms. For instance, if lectures were reviewed negatively at Level 2, 
an open-ended questionnaire about the lectures could be developed and 
used. Alternatively^ the faculty member could engage in discussions 
with his students concerning improvement of the lectures. The 
important contributions to Level 3 evaluation which colleagues and 
evaluation specialists can make are examined. (Author/KM) 
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SYMPOSIUM OVERVIEW 



The purpose of this symposium on the topic, "A Plan for the Comprehensive 
Evaluation of College Teaching," was to present and discuss a 3x3 matrix of inputs 



and techniques that could be used as a plan for the evaluation and improvement ot 
college teaching. A schema of the matrix would look something like this: 



Students 



INPUTS FROM: 

Administrators 
Faculty & Specialists 



Level 1 



Level 2 



Level 3 



PURPOSES 



Provide data for university- 
wide comparison of instruc- 
tors and courses. 

Provide data for comparative 
purposes within teaching 
units and to identify problem 
areas in instruction and 
courses. 

Provide data to pinpoint 
reasons for problems 
identified by Level 2 
evaluation. 



The details of the plan yyere presented in papers by the six participants. An 
overview was provided by H. Richard Smock and Terence J. Crooks in "A Plan for 
the Comprehensive Evaluation of College Teaching." Gerald Gillmore discussed a 
method to gather valid and reliable data at Level i in "Evaluation by Students for 



University-Wide Comparative Purposes." Level 2 was covered by Barak Rosenshine 
in "Faculty and Administrative Inputs to Instructional Evaluation" and Lawrence 
Aleamoni in "Evaluation by Students to Identify General Instructional Problems." 
Level 3 inputs were described by Keith Wharton in "Analyzing and Improving 
Instructional Practices." The overall approach was discussed by Wilbert J. 
McKeachieand C.H. Roid. 



ANALYZING AND IMPROVING INSTRUCTIONAL PRACTICES 



Introduction 

The keys to the improvement of instructional practices in American colleges 

and universities are solidly in the hands of individual teachers. Administrators may 

urge improvement, colleagues may support improvement, students may demand 

improvement, but until the teacher in the classroom decides to act, nothing will 

TiappeiT. And these teachers are extremely busy people. Not only do t;*ey have 

classes to teach — which means lectures to prepare, demonstrations and laboratories 

to arrange, examinations to construct and grade, etc. — they also have students to 

advise, committees to serve on, research to do, and papers and books to write. 

They read a great deal, travel a lot, and tend to become actively involved in a 

number of social and community activities. 

I call attention to the obvious only because of the implications that follow for 

those of us who advocate evaluation of instruction for improvement of instruction. We 

must remember that the programs we design and the procedures we recommend are for 

people who have very little time to devote to them. In some ideal world we might 

expect all teachers to give top priority to evaluation schemes that include detailed 

diagnoses of problems and recommendations for solutions (many of which of 

necessity would require teachers to go somewhere and participate in some activity, 

e.g. , a series of workshops on the preparation of instructional objectives) that no 

doubt Would improve their teaching. In our real world, however, we must realize 

that such schemes, which should be developed and made available to all teachers, 
will be utilized by only a few, and even these few may not consider them the most 
important activities in their lives. 
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With these thoughts in mind, I will attempt to complete the remaining cells 
of the 3 X 3 evaluation matrix described by Richard Smock and Terence Crooks. 
This third level of the matrix will focus on methods of getting specific feedback data 
from students and colleagues and on suggestions for providing assistance to improve 
instruction. It is assumed that the individual teacher for whom these procedures 
are recommended: 1) has utilized Level 2 evaluations to secure inputs from students 
and colleagues to identify general areas in courses and instruction in which improv- 

— ement is needed, and 2) sincerely wishes to use these^1Kpurs^to~Tmprt)ve-mstru 

and IS willing to devote at least a minimal amount of time and effort to doing so. 

Student Input 

instruction, in modern institutions cf higher education, occurs In a variety 
of ways under a multitude of conditions. There are large lecture sessions, small 
group discussions, closed-circuit television presentations, laboratory experiments, 
rigidly-prescribed activities, independent study contracts, and almost endless 
combinations of these options plus many others as well . No one course or 
instructor could ever utilize all of the possible approaches to teaching, obviously, 
but most use more than one. This means that any teacher who wishes to use 
student input to evaluate and improve instruction must be prepared to secure these 
inputs about each facet of the teaching-learning situation. And since courses, 
students, and teachers change from term to term, whatever means are developed to 
gather these inputs cannot be "permanent," but must be made flexible and open 
enough to apply to the altered situations. 

-2- 
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The task is not as difficult as it may appear at first glance, however. The 
Level 2 evaluation will have identified the area or areas of instruction in need of 
improvement. Such things as organization of the course material, communication 
and interaction skills of the instructor, examination and evaluation procedures, or 
amount and difficulty of required work may have been included. The teacher's 
responsibility at Level 3 \i now to follow up this preliminary diagnosis with 
procedures and questions designed to uncover the specific problems, and to 
determine wRaf can iiisririnrtf>-fo-hcing t^Hnifi jmprovfimpnt. Four guidelines for 
accomplishing this are offered: 

1. The opportunity to provide input for Level 3 evaluations should be 
made available to each student, but input should not be required 
from any. Although Level 2 evaluations may have identified the 
areas in which students in a class are dissatisfied and would like 
to see improvement, it is unlikely that every student is having 
difficulty with every problem area identified. It does not seem 
wise, therefore, to attempt to force a response from students who 
really have-no complaint. The results of such an effort might 
yield data that would only obscure the actual problem. 

2. Input from students should be solicited only in those areas in which 
change is possible. There is no need to further frustrate students 
by asking them to offer suggestions about situations which can not 
or will not be altered. If realities dictate, for example, that a 
course be taught by closed-circuit television and Level 2 evaluations 



show that the students are not pleased with this method, there is no 
need to ask students at Level 3 if they would prefer small-group 
discussions. A more profitable alternative would be to try to 
locate specific shortcomings of the televised presentations and 
attempt to correct them . 

3. The data-gathering instruments used should be simpie, easy to 
administer, and require only a small amount of class time to complete. 

. _ _ Teaching and learning are the main ingredients of a class^aod. ari__ 

inordinate amount of time and effort devoted to copfiplex evaluation 
of instruction procedures seems completely unjustifiable. 

4. Whenever possible, follow-up procedures should provide for 
discussions between teacher and students - Questionnaires can 
uncover some problems, but others can be ferreted out only in 
person-to-person conversations. 

The incorporation of these guidelines into procedures for securing student 
input into Level 3 evaluation is a fairly straightforward process: the students 
must be asked direct questions about the areas of instruction identified as in need 
of improvement. The- exact method used will depend upon a number of factors 
the personality of the teacher, the number of students in the class, the nature of the 
course, etc. — but the matter must be taken to the students in every instance. 
Perhaps two examples illustrating how this may be done will be helpful. 

Suppose, for instance, that a teacher of Beginning Spanish (30 students in a 
class) learns from the Level 2 evaluations that her students are dissatisfied with the 



oral drills that are a part c^f every class period. If she has established sufficient 
rapport with at least some of the students in the class, all that might be needed to 
uncover the specific complaints is a brief discussion of the drills with a few of these 
students. This could take place informally outside of class hours. If she wishes to 
give every student an opportunity to respond formally, she might prepare a brief, open- 
ended questionnaire about the oral drills and use the last five or ten minutes of a class 
period to administer it. The questionnaire could look something like this: 

(See^Appiiiiatx-t} — ^ — ^ , . . 

A combination of these two approaches can be used quite effectively in large 
classes such as an Introductory course in Sociology. A short questionnaire, 
similar to the one just mentioned, can be used to gather specific feedback data from 
the students about the area of instruction in question. The results can then be 
discussed by the teacher with a volunteer committee of students from the class. In 
this way every student ~ even In a clas§ of 250 students — will have a chance to 
express his criticisms and offer suggestions for improvement, and the teacher will 
be able to "go beyond the data" through the open discussions. 

Colleague Input 

A teacher's colleagues can play an important role in Level 3 evaluation if the 
teacher seeking help is careful not to "wear out his welcome." It would, no doubt, 
be expecting too much to ask a fellow faculty member who had assisted a teacher at 
Level 2 by reviewing and commenting upon course outlines, objectives, assignments, 
examinations, etc. , to now sit through every one of the teacher's class sessions for 
a term and identify each weakness observed in lecturing. It does not seem unreasonable. 
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however, to ask this colleague to visit the class for twenty minutes some day while 
the teacher is conducting a class activity that has been identified at Level 2 as 
needing improvement, and later commenting upon what was noticed. 

To continue one of the examples given earlier, the Spanish teacher having 
difficulty with oral drills could invite one of her colleagues to observe the oral rjril! 
portion of her class. Before the visit, however, the colleague should receive a 
description of the procedures that are customarily followed in the drills and a 
statement of whaFfPTe Teairher^epes- to a c c qmftUsh4>y them . (An example is given in 
Appendix 2.) This will enable that person to develop criteria against which to compa 
and evaluate what is actually observed. (See example in Appendix 3J During the 
visit, the observer might note that the teacher is calling on only the students who 
can readily respond, and quickly skips over or ignores those having difficulty (when 
the drills were intended to give every student practice in answering and asking 
questions); or is habitually "speaking to the chalkboard," making it difficult for the 
students to hear what is said; or any number of things that might help pinpoint the 
specific problems. A subsequent discussion of the observations could enable the 
Spanish teacher to make corrections that would convert the oral drills into one of the 
high points of the class for her students. 

There are any number of ways that colleagues can provide input into Level 3 
evaluations. Class visitations are only one. Probably one of the most profitable is 
simply a frank discussion of the particular activity in question with others who also 
are interested in improving their teaching. The arrangements for such discussions 
can be simplified if an entire department or a significant number of teachers from 
a department participates in the three-level evaluation plan that is being proposed 



at this symposium. If this is done, it is quite probable that Level 2 evaluations will 
uncover problem areas that are common to several teachers. These teachers could 
then work together, with the aid of specialists, where needed, to identify the specific 
difficulties that each is having and find solutions that are acceptable to each individual. 

There are two key factors to be kept in mind when requesting colleague aid 
with Level 3 evaluations: 1) the request should be for a small amount of time, and 
2) it should call for help with a specified area of concern. Sending one's entire 
package of course objectives, lecture notes, examinations, and a course syllabus to 
a teaching friend with the request to "Let me know if you see anything wrong." 
is not likely to yield useable results. 

Specialist Input 

Specialist input into evaluationiTf^tnstruction is most often thought of as occuring 
at what we have identified here as Level 1 and Level 2 evaluations. (By "specialists" 
I am referring to those persons in service organizations within colleges and univer- 
sities whose primary mission is to provide support and special services for the 
faculty of the institution. This includes such organizations as offices of institutional 
research, measurement and evaluation services centers, instructional resources 
centers, etc.) And indeed it would be impossible to mount university-wide evaluation 
efforts such as the program at the University of Illinois without the assistance of these 
persons and their organizations. 

Their services, however, in order to encourage improvement and effect change 
in instructional practices, should extend into Level 3 as well, for they possess 
expertise in solving instructional problems not found in the faculty at large. How 
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many faculty members, for instance, have been taught how to perform or use the 
results of an item analysis, or write measurable instructional objectives? How many 
have studied theories of learning or motivation? The answer^ of course^ is "Not very 
many" and no one is at all suprised, for these topics are seldom part of a graduate 
program except in Education or Psychology. Yet significant numbers of these 
teachers will learn from Level 2-type evaluations that tests, course objectives, and 
course organization to capture and maintain student interest and maximize learning 
are the areas in which improvement is most sorely needed. 

If the Instructional specialists do not step in at this point, use their special 
skills to help the teacher identify specific instructional problems, and then assist in 
finding solutions to the problems, the entire evaluation effort is in great danger of 
becoming only an exercise in futility* The situation might be roughly analogous to 
that of a person who is repeatedly told by his physician that he is sick, but is never 
told what the exact problem is or what he can do to get well . 

How can instructional specialists assist in Level 3 evaluation? The answer is 
simple to give, yet often very difficult to accomplish: take their services to the 
individual teacher . The specialist who sits in his office and waits for the faculty to 
come to him will have a long, lonely wait. As was mentioned at the beginning of this 
paper, teachers are busy people. Even the best"intentioned will find so many demands 
on their time that they will not be able to seek out the help that they need. It must be 
brought to them. 

Some general suggestions and a few examples as to how this may be done 
are given below: 
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1 . The faculty must be informed as to what services are available to 
them. Every institution has a number of service organizations, yet 
frequently no concentrated effort is made to advertise their existence. 
Merely listing them in the Student-Staff Directory is certainly 

not enough. One good way to accomplish this is to prepare a guide 
to the services aval la* j nd each faculty member a copy. An 
example is the booklet, "Instructional Resources," prepared by the 
Center for Educational Development at the University of Minnesota. 
A copy of this booklet was mailed to each faculty member on the Twin 
Cities Campus in the fall of 1972. (Copy attached) 

The guide can be supplemented by periodic "special notices" 
from individual organizations who wish to call faculty attention to 
changes in services offered, programs of special interest, etc. 

2. Some unit central to the Level 2 evaluation program should compile 
a listing of those areas of instruction most frequently identified as 

in need of improvement. If it can be done without violating the 
confidentiality of the data, this same unit should also compile the 
names of the teachers having common problems. This compilation 
could be done at the institutional level by a measurement services 
center, or at the collegial or departmental level by some administrative 
or faculty person or organization. 

The information gathered should be made available to the appropriate 
service organizations to enable them to plan and coordinate efforts to 
provide the proper assistance where most needed. The result could 
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be such activities as a mailing of "Testing Tips" from an 
evaluatiofi services office to teachers having problems with testing, 
or the scheduling of a short seminar on discussion techniques led by 
a member of the Communications faculty for teachers having difficulty 
in leading seminars. 
3. Instructional specialists should attempt to "work themselves cut of 
a job" by imparting their skills to certain members of the teaching 
faculty and then letting these teachers provide "specialist" help to 
their colleagues. In almost every department there are individuals 
who take a special interest in instructional improvement. They learn 
about and try alternative approaches to teaching, they participate in 
workshops and seminars on instruction, and they indicate in various 
other ways that teaching and its improvement is of major 
importance to them. These teachers must be made an integral part 
of any instructional improvement scheme, for there simply are not 
enough specialists to reach every member of the faculty of an 
institution and give them the individual attention that is needed. 
Specialists can, however, concentrate their efforts on these "key" 
people in the departments, teach them the skills that are needed, and 
then trust them to assist their fellows. 

A program using this approach was being planned (and is probably 
in operation at this time) by the Centre for Learning and Development 
at McGill University. A small number of teachers were to be invited 
to take part in an intensive program on the evaluation of student 
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learning sponsored by the Centre. One requirement for participation 
in the program, however, was that each participant had to agree to 
pass along what he had learned to a fellow member of his department. 
In this way the evaluation services and expertise of the Centre were 
to be spread to many faculty members, and a number of departments 
were to acquire their own "in-house" expert on evaluation. 

Summary 

This paper on the third level of a three-level plan for the comprehensive 
evaluation of college teaching has explored several approaches to securing 
student and colleague input into the identification and solution of specific 
instructional problems. It has also dealt briefly with some of the ways specialists 
can aid in the instructional improvement efforts of individual teachers. It was 
emphasized that teachers, who ultimately determine whether or not teaching will 
improve, have very little time to devote to formal instructional improvement 
activities and programs. Any scheme, therefore, for the evaluation and improvement 
of teaching must take this fact into consideration and be designed so that services 
are brought to the teacher rather than requiring that the teacher devote any 
significant amount of time to seeking them out. 
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Appendix 1 



Spanish 1-102 
Section ^ 
February 14, 1973 



The results of the Student Reactions to Instruction survey that you 
completed last week showed that improvement is needed in our oral drills. 
Please help me pinpoint the problem. Be as specific as you can. 

1 . What problems are you having with the oral drills? 



2. What can be done to correct these problems? 



Thanks — 
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February 16, 1973 



Dear John: 

Thanks for agreeing to help me find the causes for the troubles Mm having 
with the oral drills. I'll list below what we customarily do in the drills and what 
I hope to accomplish by them. This should help you know what to expect when 
you visit the class next Thursday . 



Purpose of Drills 

The purpose of the drills is to give every student in the class an opportunity 
to both hear and speak Spanish. They are conducted at normal speaking speed, 
and I want the students to respond correctly without hesitation. 



Class Procedures 

1. The students will have been assigned a short basic dialogue, paragraph 
or reading selection to study. 

2. I will begin by reviewing the assignment and answering any questions 
about neA/ materials. 

3. I will then ask a student a question based on the assignment. I expect 
a complete answer, not just a single-word reply, 

^. I will proceed by asking another question of some other student, or by 
asking the first student to direct a question to one of his classmates, 

5. This basic pattern will be repreated throughout the drills. At the end 
I will summarize and again answer any questions the students have. 
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Appendix 3 
Check-list For Class Visit 

1 . Were students given an assignment to study? 

2. Did teacher review assignment and answer questions? 

3. Did teacher use normal speaking speed? 

4. Did teacher speak clearly and loudly enough to be heard? 

5. Did every student have an opportunity to both answer and ask at least 
one question? 

6. Were responses given without hesitation and as a complete reply? . 

7. Did teacher summarize and answer question^: at the conclusion of the 
oral drills? 

8. Comments: 
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